Scalable Surface Reconstruction from Point Clouds with Extreme Scale and Density Diversity

In this paper we present a scalable approach for robustly computing a 3D surface mesh from multi-scale multi-view stereo point clouds that can handle extreme jumps of point density (in our experiments three orders of magnitude). The backbone of our approach is a combination of octree data partitioning, local Delaunay tetrahedralization and graph cut optimization. Graph cut optimization is used twice, once to extract surface hypotheses from local Delaunay tetrahedralizations and once to merge overlapping surface hypotheses even when the local tetrahedralizations do not share the same topology.This formulation allows us to obtain a constant memory consumption per sub-problem while at the same time retaining the density independent interpolation properties of the Delaunay-based optimization. On multiple public datasets, we demonstrate that our approach is highly competitive with the state-of-the-art in terms of accuracy, completeness and outlier resilience. Further, we demonstrate the multi-scale potential of our approach by processing a newly recorded dataset with 2 billion points and a point density variation of more than four orders of magnitude - requiring less than 9GB of RAM per process.Comment: This paper was accepted to the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. The copyright was transfered to IEEE (ieee.org). The official version of the paper will be made available on IEEE Xplore (R) (ieeexplore.ieee.org). This version of the paper also contains the supplementary material, which will not appear IEEE Xplore (R

Using Self-Contradiction to Learn Confidence Measures in Stereo Vision

Learned confidence measures gain increasing importance for outlier removal and quality improvement in stereo vision. However, acquiring the necessary training data is typically a tedious and time consuming task that involves manual interaction, active sensing devices and/or synthetic scenes. To overcome this problem, we propose a new, flexible, and scalable way for generating training data that only requires a set of stereo images as input. The key idea of our approach is to use different view points for reasoning about contradictions and consistencies between multiple depth maps generated with the same stereo algorithm. This enables us to generate a huge amount of training data in a fully automated manner. Among other experiments, we demonstrate the potential of our approach by boosting the performance of three learned confidence measures on the KITTI2012 dataset by simply training them on a vast amount of automatically generated training data rather than a limited amount of laser ground truth data.Comment: This paper was accepted to the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. The copyright was transfered to IEEE (https://www.ieee.org). The official version of the paper will be made available on IEEE Xplore (R) (http://ieeexplore.ieee.org). This version of the paper also contains the supplementary material, which will not appear IEEE Xplore (R

Prioritized Multi-View Stereo Depth Map Generation Using Confidence Prediction

In this work, we propose a novel approach to prioritize the depth map computation of multi-view stereo (MVS) to obtain compact 3D point clouds of high quality and completeness at low computational cost. Our prioritization approach operates before the MVS algorithm is executed and consists of two steps. In the first step, we aim to find a good set of matching partners for each view. In the second step, we rank the resulting view clusters (i.e. key views with matching partners) according to their impact on the fulfillment of desired quality parameters such as completeness, ground resolution and accuracy. Additional to geometric analysis, we use a novel machine learning technique for training a confidence predictor. The purpose of this confidence predictor is to estimate the chances of a successful depth reconstruction for each pixel in each image for one specific MVS algorithm based on the RGB images and the image constellation. The underlying machine learning technique does not require any ground truth or manually labeled data for training, but instead adapts ideas from depth map fusion for providing a supervision signal. The trained confidence predictor allows us to evaluate the quality of image constellations and their potential impact to the resulting 3D reconstruction and thus builds a solid foundation for our prioritization approach. In our experiments, we are thus able to reach more than 70% of the maximal reachable quality fulfillment using only 5% of the available images as key views. For evaluating our approach within and across different domains, we use two completely different scenarios, i.e. cultural heritage preservation and reconstruction of single family houses.Comment: This paper was accepted to ISPRS Journal of Photogrammetry and Remote Sensing (https://www.journals.elsevier.com/isprs-journal-of-photogrammetry-and-remote-sensing) on March 21, 2018. The official version will be made available on ScienceDirect (https://www.sciencedirect.com

Der Einsatz unbemannter Flugsysteme zur Charakterisierung von gesprengtem Haufwerk

Die erreichte Zerkleinerung und die Form des Haufwerks sind die beiden wichtigsten Ergebnisse einer Tagebausprengung. Schnelle Informationen über die Eigenschaften des gesprengten Haufwerks ermöglichen eine zielgerichtete und effiziente Produktionsplanung und Kenntnisse über die erreichte Zerkleinerung ermöglichen außerdem Anpassungen in der weiteren Zerkleinerungskette. Durch den Einsatz von UAVs (unmanned aerial vehicles) gemeinsam mit modernen Algorithmen aus dem Bereich Computer Vision und des maschinellen Lernens soll eine schnelle Erfassung und Interpretation der Daten bei gleichzeitiger Integration in die herkömmlichen betrieblichen Abläufe ermöglicht werden, und außerdem können Schwächen bodengebundener Systeme hinsichtlich Vollständigkeit und Repräsentativität umgangen werden. Im vorliegenden Beitrag wird einerseits auf den relevanten Stand des Wissens und der Technik eingegangen und andererseits wird die verfolgte Stoßrichtung bei der Systementwicklung dargelegt sowie erste Arbeiten präsentiert.The fragmentation and the shape of the muck pile are the two major outcomes of open pit mine and quarry blasts. Fast information about the muck pile properties will help to improve the production scheduling and furthermore this information could be used to optimize the blasting patterns of future production blasts. The combined use of unmanned aerial vehicles (UAVs) and modern machine learning and computer vision systems offers a new way of acquiring spatial data to determine on-site fragment size distribution, while at the same time enabling integration into common work flows and mitigating the weaknesses of ground-based systems with special regard to completeness and representativeness. In the present paper, we will discuss the relevant related work, present the planned path for system development and give examples of first work

Using Self-Contradiction to Learn Confidence Measures in Stereo Vision

Learned confidence measures gain increasing impor- tance for outlier removal and quality improvement in stereo vision. However, acquiring the necessary training data is typically a tedious and time consuming task that involves manual interaction, active sensing devices and/or synthetic scenes. To overcome this problem, we propose a new, flexi- ble, and scalable way for generating training data that only requires a set of stereo images as input. The key idea of our approach is to use different view points for reason- ing about contradictions and consistencies between multi- ple depth maps generated with the same stereo algorithm. This enables us to generate a huge amount of training data in a fully automated manner. Among other experiments, we demonstrate the potential of our approach by boost- ing the performance of three learned confidence measures on the KITTI2012 dataset by simply training them on a vast amount of automatically generated training data rather than a limited amount of laser ground truth data

UAV-based Autonomous Image Acquisition with Multi-View Stereo Quality Assurance by Confidence Prediction

In this paper we present an autonomous system for acquiring close-range high-resolution images that maximize the quality of a later-on 3D reconstruction with respect to coverage, ground resolution and 3D uncertainty. In contrast to previous work, our system uses the already acquired images to predict the confidence in the output of a dense multi-view stereo approach without executing it. This confidence encodes the likelihood of a successful reconstruction with respect to the observed scene and potential camera constellations. Our prediction module runs in real-time and can be trained without any externally recorded ground truth. We use the confidence prediction for on-site Quality assurance and for planning further views that are tailored for a specific multi-view stereo approach with respect to the given scene. We demonstrate the capabilities of our approach with an autonomous Unmanned Aerial Vehicle (UAV) in a challenging outdoor scenario

Prioritized multi-view stereo depth map generation using confidence prediction

In this work, we propose a novel approach to prioritize the depth map computation of multi-view stereo (MVS) to obtain compact 3D point clouds of high quality and completeness at low computational cost. Our prioritization approach operates before the MVS algorithm is executed and consists of two steps. In the first step, we aim to find a good set of matching partners for each view. In the second step, we rank the resulting view clusters (i.e. key views with matching partners) according to their impact on the fulfillment of desired quality parameters such as completeness, ground resolution and accuracy. Additional to geometric analysis, we use a novel machine learning technique for training a confidence predictor. The purpose of this confidence predictor is to estimate the chances of a successful depth reconstruction for each pixel in each image for one specific MVS algorithm based on the RGB images and the image constellation. The underlying machine learning technique does not require any ground truth or manually labeled data for training, but instead adapts ideas from depth map fusion for providing a supervision signal. The trained confidence predictor allows us to evaluate the quality of image constellations and their potential impact to the resulting 3D reconstruction and thus builds a solid foundation for our prioritization approach. In our experiments, we are thus able to reach more than 70% of the maximal reachable quality fulfillment using only 5% of the available images as key views. For evaluating our approach within and across different domains, we use two completely different scenarios, i.e. cultural heritage preservation and reconstruction of single family houses

Evaluations on multi-scale camera networks for precise and geo-accurate reconstructions from aerial and terrestrial images with user guidance

During the last decades photogrammetric computer vision systems have been well established in scien- tific and commercial applications. Recent developments in image-based 3D reconstruction systems have resulted in an easy way of creating realistic, visually appealing and accurate 3D models. We present a fully automated processing pipeline for metric and geo-accurate 3D reconstructions of complex geome- tries supported by an online feedback method for user guidance during image acquisition. Our approach is suited for seamlessly matching and integrating images with different scales, from different view points (aerial and terrestrial), and with different cameras into one single reconstruction. We evaluate our ap- proach based on different datasets for applications in mining, archaeology and urban environments and thus demonstrate the flexibility and high accuracy of our approach. Our evaluation includes accuracy related analyses investigating camera self-calibration, georegistration and camera network configuration

Der Einsatz unbemannter Flugsysteme zur Charakterisierung von gesprengtem Haufwerk

Die erreichte Zerkleinerung und die Form des Haufwerks sind die beiden wichtigsten Ergebnisse einer Tagebausprengung. Schnelle Informationen über die Eigenschaften des gesprengten Haufwerks ermöglichen eine zielgerichtete und effiziente Produktionsplanung und Kenntnisse über die erreichte Zerkleinerung ermöglichen außerdem Anpassungen in der weiteren Zerkleinerungskette. Durch den Einsatz von UAVs (unmanned aerial vehicles) gemeinsam mit modernen Algorithmen aus dem Bereich Computer Vision und des maschinellen Lernens soll eine schnelle Erfassung und Interpretation der Daten bei gleichzeitiger Integration in die herkömmlichen betrieblichen Abläufe ermöglicht werden, und außerdem können Schwächen bodengebundener Systeme hinsichtlich Vollständigkeit und Repräsentativität umgangen werden. Im vorliegenden Beitrag wird einerseits auf den relevanten Stand des Wissens und der Technik eingegangen und andererseits wird die verfolgte Stoßrichtung bei der Systementwicklung dargelegt sowie erste Arbeiten präsentiert.The fragmentation and the shape of the muck pile are the two major outcomes of open pit mine and quarry blasts. Fast information about the muck pile properties will help to improve the production scheduling and furthermore this information could be used to optimize the blasting patterns of future production blasts. The combined use of unmanned aerial vehicles (UAVs) and modern machine learning and computer vision systems offers a new way of acquiring spatial data to determine on-site fragment size distribution, while at the same time enabling integration into common work flows and mitigating the weaknesses of ground-based systems with special regard to completeness and representativeness. In the present paper, we will discuss the relevant related work, present the planned path for system development and give examples of first work

Der Einsatz unbemannter Flugsysteme zur Charakterisierung von gesprengtem Haufwerk

Die erreichte Zerkleinerung und die Form des Haufwerks sind die beiden wichtigsten Ergebnisse einer Tagebausprengung. Schnelle Informationen über die Eigenschaften des gesprengten Haufwerks ermöglichen eine zielgerichtete und effiziente Produktionsplanung und Kenntnisse über die erreichte Zerkleinerung ermöglichen außerdem Anpassungen in der weiteren Zerkleinerungskette. Durch den Einsatz von UAVs (unmanned aerial vehicles) gemeinsam mit modernen Algorithmen aus dem Bereich Computer Vision und des maschinellen Lernens soll eine schnelle Erfassung und Interpretation der Daten bei gleichzeitiger Integration in die herkömmlichen betrieblichen Abläufe ermöglicht werden, und außerdem können Schwächen bodengebundener Systeme hinsichtlich Vollständigkeit und Repräsentativität umgangen werden. Im vorliegenden Beitrag wird einerseits auf den relevanten Stand des Wissens und der Technik eingegangen und andererseits wird die verfolgte Stoßrichtung bei der Systementwicklung dargelegt sowie erste Arbeiten präsentiert.The fragmentation and the shape of the muck pile are the two major outcomes of open pit mine and quarry blasts. Fast information about the muck pile properties will help to improve the production scheduling and furthermore this information could be used to optimize the blasting patterns of future production blasts. The combined use of unmanned aerial vehicles (UAVs) and modern machine learning and computer vision systems offers a new way of acquiring spatial data to determine on-site fragment size distribution, while at the same time enabling integration into common work flows and mitigating the weaknesses of ground-based systems with special regard to completeness and representativeness. In the present paper, we will discuss the relevant related work, present the planned path for system development and give examples of first work